Behavioral repertoire learning in robotics

Behavioral Repertoire Learning in Robotics Antoine Cully ISIR, Université Pierre et Marie Curie-Paris 6, CNRS UMR 7222 4 place Jussieu, F-75252, Paris Cedex 05, France [email protected] Jean-Baptiste Mouret ISIR, Université Pierre et Marie Curie-Paris 6, CNRS UMR 7222 4 place Jussieu, F-75252, Paris Cedex 05, France [email protected] ABSTRACT Learning in robotics typically involves choosing a simple goal (e.g. walking) and assessing the performance of each con- troller with regard to this task (e.g. walking speed). How- ever, learning advanced, input-driven controllers (e.g. walk- ing in each direction) requires testing each controller on a large sample of the possible input signals. This costly pro- cess makes difficult to learn useful low-level controllers in robotics. Here we introduce BR-Evolution, a new evolutionary learn- ing technique that generates a behavioral repertoire by tak- ing advantage of the candidate solutions that are usually discarded. Instead of evolving a single, general controller, BR-evolution thus evolves a collection of simple controllers, one for each variant of the target behavior; to distinguish similar controllers, it uses a performance objective that al- lows it to produce a collection of diverse but high-performing behaviors. We evaluated this new technique by evolving gait controllers for a simulated hexapod robot. Results show that a single run of the EA quickly finds a collection of controllers that allows the robot to reach each point of the reachable space. Overall, BR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot

Professional training and participatory research: Combined actions for developing organic rice farming in the Camargue region of France

In 2006 and 2007, INRA’s Joint Research Unit, Innovation, was a partner in a European professional training project within the framework of the Leonardo da Vinci programme. The objective of this project was to help develop organic rice farming in the major European rice-growing regions where rice is mainly cultivated in ecologically-sensitive areas. In France, the rate of conversion to organic production is much lower that what would be expected, since organic rice farming presents particular technical problems. The availability of expert support is critical to successful conversion and no structured training was available in the past. This is the reason why we developed a participatory training method that helps rice growers and stakeholders to convert to organic farming and to improve their organic rice production. Different training sessions were organised. The participants shared their thoughts about technical problems encountered and identified possible solutions. Some of the topics developed were weeds, soils and fertility, and varieties. At the end of these sessions, a motivated workgroup was set up. Some of its members even proposed to assess the efficiency of some of the techniques that were discussed during the work sessions in fields on their own farms. Furthermore, field visits were organised in the Camargue region of France and in Spain. Scientists and group members hope to be able to continue to work together after the O.R.P.E.S.A. project is over. In order to make this possible, we are now planning to initiate new research and development actions using the same approach

Quality-diversity optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning

Quality-diversity optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning

IOP PUBLISHING

Artificial evolution of the morphology and kinematics in a flapping-wing mini-UA

Open-Ended Evolutionary Robotics: an Information Theoretic Approach

This paper is concerned with designing self-driven fitness functions for Embedded Evolutionary Robotics. The proposed approach considers the entropy of the sensori-motor stream generated by the robot controller. This entropy is computed using unsupervised learning; its maximization, achieved by an on-board evolutionary algorithm, implements a "curiosity instinct", favouring controllers visiting many diverse sensori-motor states (sms). Further, the set of sms discovered by an individual can be transmitted to its offspring, making a cultural evolution mode possible. Cumulative entropy (computed from ancestors and current individual visits to the sms) defines another self-driven fitness; its optimization implements a "discovery instinct", as it favours controllers visiting new or rare sensori-motor states. Empirical results on the benchmark problems proposed by Lehman and Stanley (2008) comparatively demonstrate the merits of the approach

Local adaptations of Mediterranean sheep and goats through an integrative approach

Small ruminants are suited to a wide variety of habitats and thus represent promising study models for identifying genes underlying adaptations. Here, we considered local Mediterranean breeds of goats (n = 17) and sheep (n = 25) from Italy, France and Spain. Based on historical archives, we selected the breeds potentially most linked to a territory and defined their original cradle (i.e., the geographical area in which the breed has emerged), including transhumant pastoral areas. We then used the programs PCAdapt and LFMM to identify signatures of artificial and environmental selection. Considering cradles instead of current GPS coordinates resulted in a greater number of signatures identified by the LFMM analysis. The results, combined with a systematic literature review, revealed a set of genes with potentially key adaptive roles in relation to the gradient of aridity and altitude. Some of these genes have been previously implicated in lipid metabolism (SUCLG2, BMP2), hypoxia stress/lung function (BMPR2), seasonal patterns (SOX2, DPH6) or neuronal function (TRPC4, TRPC6). Selection signatures involving the PCDH9 and KLH1 genes, as well as NBEA/NBEAL1, were identified in both species and thus could play an important adaptive role

Neurogenesis Drives Stimulus Decorrelation in a Model of the Olfactory Bulb

The reshaping and decorrelation of similar activity patterns by neuronal networks can enhance their discriminability, storage, and retrieval. How can such networks learn to decorrelate new complex patterns, as they arise in the olfactory system? Using a computational network model for the dominant neural populations of the olfactory bulb we show that fundamental aspects of the adult neurogenesis observed in the olfactory bulb -- the persistent addition of new inhibitory granule cells to the network, their activity-dependent survival, and the reciprocal character of their synapses with the principal mitral cells -- are sufficient to restructure the network and to alter its encoding of odor stimuli adaptively so as to reduce the correlations between the bulbar representations of similar stimuli. The decorrelation is quite robust with respect to various types of perturbations of the reciprocity. The model parsimoniously captures the experimentally observed role of neurogenesis in perceptual learning and the enhanced response of young granule cells to novel stimuli. Moreover, it makes specific predictions for the type of odor enrichment that should be effective in enhancing the ability of animals to discriminate similar odor mixtures